Psychological and educational assessment tools have been used for the measurement and evaluation of abilities, aptitudes, personality characteristics and so forth. An early example of such tests dates back to 1904 when psychologist Alfred Binet was asked by the French Ministry of Education to design a test that would identify slow learners. It was noted that some children did not learn well in an ordinary classroom setting and needed special help. Reluctant to let teachers identify slow learners due to the possibility of prejudicial bias, an objective test was sought that would reveal who could benefit from remedial work. Binet's brainstorm was the famous IQ test.
It is well to note the basic difference between the assessment tools that are the subject of this invention and the skill acquisition or remedial training tools commonly known as computer-assisted instruction (CAI). Many examples of the latter courseware for simple presentation or the newer multi-media versions are available in subjects from art to zoology. The main attribute of such courseware (including testing) is the interactive nature of the programs and the convenience and "customizing" of each session as a function of the demonstrated skill level of the student. In CAI the notions of reliability, validity, bias, and standardization are seldom of central importance because of the objective nature of the courseware.
These notions, however, are of primary importance in the psychological and educational assessment tools; furthermore, accurate determination of the subject's basals and ceilings is required for the proper administration and scoring of the test.
The allocation of presentation material to video vs. audio is just a matter of style or perceived effectiveness in the CAI environment. The situation is quite different regarding assessment tools which are rigorously field tested (ie. "normed") for reliability and validity; the mode of administration is precisely stipulated, and any deviation may violate the conditions under which it had been normed. For psychological and educational assessment tools, the full range of multi-media presentation features closely simulates the manual version of a normed test. The standards used to create the manual test are easily translated to the computer test by a comparative study of the similarity of test scores of computer assisted versus manual testing.
To summarize the distinctions between CAI and norm referenced computer-assisted assessment (CAA) of the present invention, the uses of each as well as the scoring techniques should be explored. CAI is used to teach new material, to review learned material as in drill and practice, to teach productivity techniques such as word processing and to teach programming. Some CAI programs have features to keep track of scores or to adjust the level of difficulty. In contrast, CAA of the present invention is an assessment tool; teaching is not the objective. Scoring of CAA is sophisticated with raw scores, standard scores, subtests, age equivalents, and basals and ceilings. The testing rules of CAA are rigid with no deviation allowed.
The traditional psychological and educational assessment manual tools are usually administered one-on-one with a clinician serving the dual role of test administrator and observer/recorder. For brain damaged adults, the clinician is typically an experienced psychologist or psychiatrist. However, in the school environment for educational or psychological assessment, the clinician may be a social worker, a visiting psychologist, a speech therapist, a reading specialist, a resource room teacher, a special education teacher, etc. While test administration requires a training program, the proficiency of the clinician is variable especially in the school environment. Each of the assessment tools has extensive instructions for the clinician in the area of question sequencing and scoring related to establishing the critical range of the subject; this is involved with the subject's basals and ceilings which are determined by runs of right or wrong answers within a particular sequence. A rule might be, "if the subject gets two questions wrong in succession, change to sequence 4 and score all of the answers below question x as wrong and above question y as right". To the uninitiated, this might seem counter intuitive; field experience has shown this aspect of test administration and scoring to be most problematic, especially for clinicians with limited assessment background. For example, clinicians tend to be reluctant to "penalize" subjects by not giving credit for correct answers above the ceiling; others show a reluctance to grant unearned credit for incorrect answers below the basal. Other reasons for bias such as social class prejudice or the assumption that shy or disruptive children are learning impaired may be operative. The subjectivity of manual testing precludes true standardized administration. Subjectivity and deviation from test rules can take place during any part of the testing session. Factors may involve the clinician or the subject.
In the traditional test session, the clinician is expected to manually administer the test, record the scores, and assess the subject's response and behavior. In addition, the clinician is expected to compute the scores and sometimes to write a report suggesting learning programs. This is difficult to do well, especially with "difficult" test subjects. In fact, such testing using a human examiner is often not possible for children with attention deficit disorder (ADD) or those too young to focus long enough to be accurately tested.
While clinicians may be very proficient in their respective fields, they may not speak clearly. They are expected to flawlessly recite scripts to the subjects during evaluation. In some cases, the subject is getting the question wrong just because he or she doesn't understand the examiner. In situations with "English as a Second Language" (ESL) subjects the problem is compounded. Sometimes, the reverse problem manifests itself if the clinician has an accent.
Computer software for test scoring and analysis of error patterns is available for manually administered tests. However, it is necessary to manually enter the raw scores into the computer prior to the automated analysis. Also, there is no computer guidance in administrating the test since this is done as an ad-hoc procedure after the test is over.